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INFORMATION  THEORETIC  APPROACH  TO  GEOMETRIC 


PROGRAMMING 

by 

P.  Brockett 
and 

A.  Chames 


Abstract 

This  paper  shows  how  the  fundamental  geometric  inequality  lemma  of  geometric 
programming  can  be  obtained  immediately  from  information  theoretic  methods.  This 
results  in  a  drastic  simplification  of  the  proof  and  points  the  way  to  other  connections 
between  information  theory  and  geometric  programming 
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Introduction 


This  note  shows  how  information  theoretic  methods  can  be  used  to  obtain  simply 
the  fundamental  geometric  inequality  lemma.  This  lemma  is  the  base  used  by  Duffin, 
Peterson  and  Zener  (1967)  for  proving  their  duality  results  for  geometric  programming, 
and  its  proof  occupies  three  pages  in  their  book.  Their  method  of  proof  is  a  technically 
impressive  procedure.  However  to  il1»strare  the  power  of  information  theoretic  methods 
we  shall  give  a  simple  proof  of  this  fundamental  inequality.  The  approach  we  shall  take  is 
to  show  the  connection  between  geometric  programming  and  constrained  Khinchine- 
Kullback-Leibler,  or  minimum  discrimination  information  (MDI)  estimation.  Complete 
duality  states  for  MDI  estimation  are  known  (c.f.  Brockett,  Chames  and  Cooper  (1980), 
Chames,  Cooper  and  Seiford  (1978),  Chames  and  Cooper  (1974a,b)).  Additionally,  in 
the  usual  duality  state  of  interest,  the  statistical  properties  of  the  solution  are  known, 
facilitating  sensitivity  analysis.  The  computation  of  MDI  estimation  is  easily  performed 
using  an  unconstrained  dual  convex  program  involving  only  linear  and  exponential 
functions. 


The  information  theoretic  approach  is  based  upon  the  mean  information  for 
discriminating  between  two  densities  fj  and  f2  (relative  to  some  fixed  dominating  measure 
X).  The  mean  information  for  discrimination  in  favor  of  fi  against  f2  is  defined  by  (c.f. 
Kullback  (1959)). 
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X(dx). 


Applying  Jensen's  inequality  to  the  convex  function  h(y)  =  ylny  with  the  random  variable 


Y  = 


f.lx) 

f2(X) 


and  assuming  that  X  has  probability  measure  f2(x)  X(dx)  yields  the  result 


2 

I(f i If2)  S:  0  with  I(filf2)  =  0  if  and  only  if  fi  =  f2  a.s.  [A.].  Amazingly,  this  is  the  only 
result  needed  to  give  a  proof  to  the  so-called  geometric  inequality  (c.f.  Duffin,  Peterson  and 
Zener  (1967)  pg  1 10).  As  usual,  OlnO  is  taken  to  be  0. 

Lemma  1  (Geometric  Inequality-Duffin,  Peterson  and  Zener  (1967)) 

Suppose  x  g  and  8  e  f  with  8;  ^  0,  i  =  1,2, . .  n. 


Then 


Ixi8i  +  (£8i)ln(X5i)  <  (ZSi)ln(Iexp{xj}>  +  ISjln  5* 


with  equality  if  and  only  if  there  exists  a  non-negative  number  B  such  that 


8i=  Bexp{x;},  i  =  1,2,. ...,n. 


Proof:  Consider  two  probability  distributions  P  and  Q  over  {1,2, . n}  given  by 

8:  ,  exp  (xj) 

Pi  =  — —  and  q;  =  - >— L  . 

Z8;  Eexpjx;] 


Then  since  I(PIQ)  >  0,  and  consequently 


0  <  (l6,)l(PIQ)  =  ISjln  jSiUexpJx^})  |  =  (z5.)ln  jx.j)  +  s5.ln5. 

5jlexp{xij)J 


(z8j)ln(z8j)  -  Z8;Xi  with  equality  if  and  only  if  p;  =  qj,  i.e.,  8j  =  Bexp(xj},  i  =  1,2, . .  n. 
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Q.E.D. 
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